Technical Field
The present invention relates to detection of a scene change in a video to enable more efficient encoding. More particularly, the invention relates to resetting a starting quantization parameter (QP) for the new scene based on the new scene complexity.
Related Art
Video commonly transitions from one scene to a different scene. The transition can take place between two consecutive video frames or multiple video frames. Normally, the transition between two frames is referred to as a scene cut and the one across multiple frames is referred as fade or dissolve.
In order to achieve good video quality, encoding scene changes properly is important. The first picture of a new scene needs to be encoded as an Intra-picture, because it cannot be efficiently inter predicted by using any frames in the previous scene as a reference frame. The first picture being encoded as an Intra-picture establishes a high-quality reference for the subsequent inter-prediction pictures in the new scene. However, regarding buffer management and rate-control, I-pictures are expensive, and encoding them at a particular location must be carefully considered to avoid excessive bit allocation by the encoder.
Additionally for a new scene, rate control models usually need to reset the starting quantization parameter (QP) based on the scene complexity. The scene complexity can be provided by a lookahead encoder in a two-pass encoding system. One way of calculating the complexity is to use multiplication of the number of encoding bits and the linear QP. The complexity can also be estimated by the spatial activity and temporal activity. The higher the complexity is, the more difficult the video is to be encoded. Generally, QP is positively related to the complexity of the video for a given bitrate. Lower QP is used for scenes with lower complexity and higher QP is used for scenes with higher complexity. Within a scene, fluctuations of QPs are small as the complexity tends to stay at the same level. When a scene change occurs, the complexity could change significantly. Therefore, encoding a scene change (SC) I-frame also requires that the rate control adjust the starting QP for the new scene.
FIG. 1 shows the complexity curve of a video transiting from a low complexity scene to a high complexity scene. The frames 1, 2, and 3 are in the low complexity scene, so they are encoded using low QPs. The frame 4 is a scene cut frame. If a similar low QP used in the old scene is used for the new scene, the scene change frame 4 would show bad quality during encoding. Therefore, the rate control needs to pick a high QP for the new scene, including during the scene change frame 4.
Error! Reference source not found. FIG. 2 shows the complexity curve of a video transiting from a high complexity scene to a low complexity scene. The frames 1 to 12 are in the high complexity scene, so they are encoded using high QPs. The frame 12 is a scene cut frame. If a similar high QP used in the old scene is used for the new scene, the scene change frame would consume a lot of unnecessary bits during encoding. Therefore, the rate control needs to pick a low QP for the new scene, including during the scene change frame 12.
The above two cases are straight forward to solve, but the following cases are not.
In FIG. 3, a first scene change is closely followed by another second scene change. The frame 4 is the first scene change frame and the frame 12 is the second one. In this example, the scene changes are 8 frames apart. Frames 1 to 3, noted as scene 1, have high complexity. Frames 4 to 11, noted as scene 2, have low complexity. Frames 12 to 16, noted as the scene 3, have medium complexity.
In one prior art encoding method, both scene changes are encoded as I frames. Low QP is selected for the scene 2 and medium QP is selected for the scene 3. The scene 2 has the lowest complexity among the three. Due to its short duration, it is not wise to spend too many bits on this scene because the Video Quality (VQ) improvement is for a short duration and not visually noticeable. Moreover, if scene 2 consumed too many bits, there could be fewer bits for the scene 3 to consume, which may lead to bad video quality in scene 3.
In another prior art method, the first scene change is ignored and the second scene is encoded as a part of scene 1. Only the second scene change at frame 12 is encoded with an I-frame having a medium QP. The purpose of this approach is to save some bits over the scene 2, but this creates a problem. The scene 1 uses a high QP. If scene 2 is encoded as a part of scene 1, it continues to use a high QP and too few bits are allocated during scene 2. This could create bad Video Quality (VQ) that is very noticeable even if scene 2 is very short.
FIG. 4 shows another example of the two close scene change case. The first scene is a low complexity scene, the second scene is short with high complexity and the third one has medium complexity. If the first scene change is ignored and encoded as part of the first scene, low QP of the scene 1 would be used for the scene 2. It would cause too many bits are used for scene 2, which may cause an encoder buffer overflow, and or compromise the ability of rate control when apportioning bits to scene 3
It is desired to provide better methods of encoding multiple successive scene changes with different QP levels without over or under allocating bits for encoding during some scenes.